This application is based on Japanese patent application No. HEI 10-313334, filed on Nov. 4, 1998, the entire contents of which are incorporated herein by reference.
a) Field of the Invention
The present invention relates to an image pickup apparatus and more particularly to an image pickup apparatus capable of picking up an image in a wide dynamic range and a method of controlling the image pickup apparatus.
b) Description of the Related Art
A solid state image pickup device has photodiodes disposed in a two-dimensional matrix shape and can take a two-dimensional image. Each photodiode corresponds to a pixel of an image.
FIG. 11 is a graph showing the photoelectric conversion characteristics of photodiodes of a solid state image pickup device. The abscissa represents the amount of light incident upon a photodiode, and the ordinate represents the voltage of a signal output from the photodiode. Characteristic curves A1, A2 and A3 show the photoelectric conversion characteristics of first, second and third photodiodes of the same solid state image pickup device.
Each of the characteristic curves A1 to A3 has a linear region R1 with a small incidence light amount and a saturated region R2 with a large incidence light amount. In the linear region R1, an output voltage is proportional to an incidence light amount. In the saturated region R1, the output voltage corresponding to incidence light is saturated.
The characteristics A1 to A3 of the photodiodes are the same in the linear region R1, whereas they are different in the saturated region R2. In the saturated region, the levels of output voltages of the photodiodes become irregular. In order to forcibly convert an output voltage of Vw or higher into a voltage Vw, a white clip process is performed.
If the white clip process is performed, the linear region R1 is only a region which can be used for photoelectric conversion. The dynamic range capable of photoelectric conversion is therefore determined basically by the width of the linear region R1.
Solid state image pickup devices are used with digital still cameras and video cameras. The dynamic range of a solid image pickup device is very narrow as compared to that of human eyes and a photographic film. A narrow dynamic range may cause white or black crushed areas in an image.
In order to avoid this, techniques are known by which an image is picked up two times at different exposure times and the two images are synthesized. The details of the techniques will be described with reference to FIGS. 12A to 12C.
Similar to FIG. 11, the abscissa of FIGS. 12A to 12C represents an incidence light amount and the ordinate represents an output voltage.
FIG. 12A is a graph showing the photoelectric conversion characteristics used by a first image pickup operation of long-time exposure. Since the exposure time is long, even if the incidence light amount per unit time is small, an output voltage is large. Therefore, the photoelectric conversion characteristics are subjected to the white clip process at a voltage Vw.
FIG. 12B is a graph showing the photoelectric conversion characteristics used by a second image pickup operation of short-time exposure. Since the exposure time is short, an output voltage for an incidence light amount per unit time is lower than that obtained by the characteristics (FIG. 12A) for the long-time exposure. The photoelectric conversion characteristics shown in FIG. 12B are also subjected to the white clip process.
FIG. 12C is a graph showing the photoelectric conversion characteristics obtained by synthesizing the first image pickup photoelectric conversion characteristics (FIG. 12A) and the second image pickup photoelectric conversion characteristics (FIG. 12B). For example, the synthesizing method is a simple addition of the two characteristics.
By using the synthesized photoelectric conversion characteristics, the dynamic range of the solid state image pickup device can be broadened. Therefore, irrespective of whether the incidence light amount is large or small, all photodiodes of a solid state image pickup device can have the uniform photoelectric conversion characteristics.
In the synthesized photoelectric conversion characteristics, a slope in the large incidence light amount region is gentler than that in the small incidence light amount region. The characteristics with different slopes are approximately equal to the human visual sense characteristics. Therefore, any practical problem will not occur even if the synthesized characteristics are used with a solid image pickup apparatus.
The first image pickup operation of long-time exposure and the second image pickup operation of short-time exposure may be performed in a reverse order.
Next, the operation of the solid image pickup device performing the above process will be described with reference to FIGS. 13 to 17. In FIGS. 13 to 17, a hatched area is an area where electric charges are stored.
FIG. 13 is a plan view of an all-pixel read type solid state image pickup device. Signals of all pixels (photodiodes) can be read to an external at the same time as one frame image.
The solid image pickup device has: photodiodes 51 disposed in a two-dimensional matrix shape for photoelectric conversion; vertical charge transfer paths (VCCD) 52 for transferring electric charges in a vertical direction; a horizontal charge transfer path (HCCD) 53 for transferring electric charges in a horizontal direction; and an output amplifier 54 for outputting a voltage corresponding to electric charges to an external.
First, as shown in FIG. 13, an image pickup operation of long-time exposure is performed to store electric charges of a first image in the photodiodes 51.
Next, as shown in FIG. 14, the electric charges of the first image stored in the photodiodes 51 are read and stored in the right side vertical charge transfer paths 52. After this data read, the first image pickup operation of long-time exposure is terminated, and an image pickup operation of short-time exposure starts for a second image.
Next, as shown in FIG. 15, the electric charges of the first image on the vertical charge transfer paths 52 are transferred downward to the horizontal charge transfer path 53. The horizontal charge transfer path 53 transfers the received electric charges from the right side to the left side to the output amplifier 54. The output amplifier 54 outputs a voltage corresponding to the received electric charges. Namely, it outputs a signal of the first image.
During this period, as shown in FIG. 16, electric charges for the second image are being stored in the photodiodes 51 by the second image pickup operation of short-time exposure which started immediately after the data read operation shown in FIG. 14.
Next, as shown in FIG. 17, the electric charges of the second image stored in the photodiodes 51 are read and stored in the right side vertical charge transfer paths 52. After this data read, the second image pickup operation of short-time exposure is terminated.
Next, similar to FIG. 15, the electric charges of the second image in the vertical charge transfer paths 52 are transferred downward to the horizontal charge transfer path 53. The horizontal charge transfer path 53 transfers the received electric charges from the right side to the left side to the output amplifier 54. The output amplifier 54 outputs a voltage corresponding to the received electric charges. Namely, it outputs a signal of the second image.
Thereafter, the first and second images are synthesized as illustrated in FIGS. 12A to 12C.
The operation of the solid state image pickup device described above does not pose any problem so long as the subject is stationary. However, if the subject is moving, the following problem occurs. Since there is a long time between the first image pickup operation of long-time exposure and the second image pickup operation of short-time exposure, the position of the subject during the first image pickup operation becomes different from that of the subject during the second image pickup operation. The synthesized image therefore has a blurred subject image.
This problem can be solved by shortening the exposure time of the second image pickup operation. The exposure of the second image pickup operation starts immediately after the data read operation shown in FIG. 14 and terminates immediately after the data read operation shown in FIG. 17. During this period, charges of the first image are transferred. Therefore, the exposure time for the second image pickup operation cannot be shortened more than the charge transfer time of the first image. The charge transfer time for one frame image is about {fraction (1/60)} to {fraction (1/15)} second.
It is an object of the present invention to provide an image pickup apparatus capable of picking an image of a moving subject in a broad dynamic range and with a high image quality.
According to one aspect of the present invention, there is provided an image pickup apparatus comprising: a plurality of photoelectric conversion elements each generating an electric signal through photoelectric conversion; a plurality of signal storage elements for storing the electric signals generated by the photoelectric conversion elements; a plurality of gates for reading the electric signals generated by the photoelectric conversion elements and storing the read electric signals in the signal storage elements; control means for performing a first image pickup operation by making the photoelectric conversion elements generate the electric signals, reading the generated electric signals and storing the read electric signals in the signal storage elements, thereafter performing a second image pickup operation under an image pickup condition different from the first image pickup operation by making the photoelectric conversion elements generate the electric signals, thereafter outputting the electric signals generated by the first image pickup operation and stored in the signal storage elements to an external, and thereafter reading the electric signals generated by the second image pickup operation from the photoelectric conversion elements, storing the read electric signals in the signal storage elements, and outputting the electric signals to the external; and synthesizing means for generating an image signal by subjecting the output electric signals generated by the first and second image pickup operations to a white clip process and synthesizing the output electric signals.
After the first and second image pickup operations are performed, the electric signals generated by the first and second image pickup operations are output to the external. The second image pickup time can be shortened without being limited by the transfer time of the electric signals generated by the first image pickup operation. Since the second image pickup time can be shortened, a blurred subject in images obtained by the first and second image pickup operations can be suppressed and the image signals of a high quality can be obtained.
Since the electric signals generated by the first and second image pickup operations are subjected to a white clip process and synthesized, the image signals can be obtained in a broad dynamic range.